1. Field of the Invention
This invention relates to microscopy, and more specifically to three-dimensional light field microscopy.
2. Description of the Related Art
FIG. 1 illustrates a prior art light field microscope that may be used for real time three-dimensional (3D) microscopy. The light field microscope is based on a conventional microscope, modified by inserting a microlens array 110 containing tens or hundreds of thousands of microlenses 112 at the intermediate image plane 120, at or about 2.5 millimeters (mm) in front of the sensor (e.g., a charge coupled device (CCD)). This microlens array transposes the light field, producing an array of, for example, 55,000 small images of the rear aperture of the objective lens 100. The images capture the directional (angular) information of the light field, while each microlens location corresponds to spatial information. In this way, the 3D light field is multiplexed onto the 2D surface of the sensor.
The microlens array 110 of FIG. 1, for example, may produce 55,000 angular images captured by the sensor at image plane 120. Each image on the sensor may be, for example 14×14 pixels, or 10×10 pixels, depending on the size of the image and the pixel spacing of the sensor. Each of the pixels in an image corresponds to one location on the objective lens. Thus, the images record information in a mixed (angular) format. Each angular image is not a complete image, and each image taken by itself is meaningless and not particularly useful. To recover the information from the images, assume that each image has a coordinate system (of pixels in the image) based on an x and y axis—e.g., 10×10. Each pixel (i,j) is taken from each image and combined. This set of pixels (i,j) from each image, when combined, will form one image, a real image of what the microscope sees. This extraction and combination is performed for each pixel in the coordinate system to produce a set of (x*y) real images (or less, as some of the pixel locations in the coordinate system, for example pixels in overlapping boundaries, may have to be discarded).
Computer-implemented software techniques, which may be equivalent or similar to those used in conventional light field rendering, tomography, and confocal microscopy, may be used to process the captured images. The software may generate 3D views, and may extract other 3D information about the observed specimen 130.
Conventional microscopes typically allow the user to select among several objectives 100 by rotating a mechanical turret. The light field microscope illustrated in FIG. 1 may be implemented via a second turret filled with microlens arrays 110 matching F/numbers of the objectives 100. This additional mechanical part adds considerable complexity to the microscope and operation thereof. Further, the microlens array 110 may need to be positioned, for example, 2.5 mm from the sensor, with precision of better than 0.1 mm regarding shifts parallel to the image plane 120. Switching from one microlens array 110 to another with the above precision and in a robust way is extremely difficult as a mechanical problem, adding to the expense of the second turret.
Each microlens array 110 may include, for example, 50,000 to 100,000 microlenses, with each microlens having a diameter of, for example, 125 microns. Such an array of microlenses is expensive and difficult to manufacture. In addition, correction for aberration would be extremely difficult, considering the size and the number of microlenses that are used.
Each microimage captures the circular aperture of the objective, and contains partially illuminated pixels at the edge of the aperture which have uncertain pixel value, and thus may have to be discarded. Further, adjacent microimages may overlap by one or two pixels, and the overlapping pixels may have to be discarded. In addition, there may be many unused pixels on the sensor not illuminated by a microlens. Thus, as much as 50%, or more, of the sensor's pixels may be wasted.
Each microimage is very small. For example, the microlens array may generate 100,000 images, with each image occupying a 10×10 pixel grid on the sensor. For the at least one of the reasons stated above, the image processing software may have to remove a 2-pixel wide border or boundary from each microimage. Thus, out of 10×10 image, only 8×8 is useful. 64 pixels is significantly less than 100 pixels. Further, the smaller the image size, the higher a percentage of pixels will be lost.